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ABSTRACT 



This paper evaluates two locally driven, school-based 



programs in Illinois that use a system of care approach to provide services 
for children and adolescents with, or who are at risk for, developing 
emotional or behavioral disturbances. These sites included a mental health 
early intervention pilot program in an elementary school setting, and a day 
treatment program established by a local school district. Students in the 
public school program or a private school program for similar students were 
evaluated at baseline and 1 year later. The majority of students were 
identified as having a primary behavioral disability; about half the youth 
had legal charges; and most common risk factors included below grade-level 
achievements and frequent suspensions /expuls ion/ truancy. No significant 
differences between students in the private day placement and the public 
special placement were found. As the early intervention program descriptive 
data indicated, the 16 children were referred for such reasons as academic 
problems, non-compliance behavior, attention difficulties, and poor peer 
interaction. Comparisons of baseline and 1 -year- later scores suggested 
substantially better functioning for all youth at Time 2. Evaluation of both 
programs supports the use of participant-driven evaluation if the data are to 
be useful for program development. (Contains 12 references.) (DB) 
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Introduction 

System of care initiatives led by mental health and child 
welfare agencies are returning youth to communities and prevent- 
ing others from being placed outside their communities (Stroul, 
1993; Epstein, Kutash, & Duchnowski, 1998). Concurrently, 
school-based mental health projects are encouraging collaborative 
partnerships and bringing social service options into school 
buildings. This has increased efforts to integrate system of care 
approaches through education programs in communities across 
the country. Changes in traditional service delivery models in 
mental health and child welfare agencies are prompting educa- 
tors to partner with social service providers in addressing the 
comprehensive needs of youth and families. Restructured 
professional roles and changes in traditional program struc- 
tures have been reported (Eber, Nelson, 1997). 

In Illinois, components of a system of care have been 
established in a variety of school-based programs over the past 
3-5 years. The Illinois State Board of Education (ISBE) has 
provided technical assistance (TA) and evaluation support to 
school districts involved in such innovations. Partnerships with 
mental health and child welfare providers have expanded the 
experiences of educators involved in school-based application 
of systems of care. 

Beginning in 1995-96 school year, school districts interested 
in TA and evaluation support related to students with, or at- 
risk of developing. Emotional or Behavioral Disturbances 
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(EBD) began collecting student/family outcome 
data. The state level TA/Evaluation team had 
assisted local stakeholders in defining evaluation 
questions and identifying components of the 
evaluation process which were appropriate for 
different sites. The intent was to support locally 
driven initiatives and design evaluation activities 
to address the questions and issues raised by the 
local stakeholders. Evaluation data have been 
collected in several sites where components of a 
system of care have been initiated for students 
with, or at-risk of, EBD. These sites included a 
mental health early intervention pilot in an elemen- 
tary school setting, and a day treatment program 
established by a local school district. This summary 
discusses Time 1 and Time 2 data for two such sites 
from Illinois. Similarities and differences across the 
early intervention and day treatment provide 
insights about the needs for comprehensive 
approaches across school settings. 

Method 

Sites 

Public school day treatment program. In 
September 1995, a school district in central Illinois 
developed a public school day treatment program 
for youth with EBD. This program was intended as 
an alternative to sending students to private day 
programs outside the school district and commu- 
nity. The intent was to improve student outcomes 
and return students more quickly and effectively to 
their neighborhood schools. Staff members were 
trained in strength-based wraparound approaches 
and incorporated these approaches into their 
individualized teams and plans for students and 
families. School-based social workers at the day 
treatment setting, who served as facilitators of 
child/family teams, were responsible for data 
collection. In addition to completing background' 
information and the CAP AS these social workers 
facilitated the completion of family, teacher, and 
youth instruments (see Table 1). 



Baseline data was collected April through May, 
1996 on 30 students. Of these students, 57% scored 
in the clinical range on the CAPAS. In spring of 
1997, the day treatment staff collected follow-up 
information on 26 of the initial 30 youth served in 
the public school day treatment program. Of those 
26, 17 had moved to a less restrictive setting while 
nine remained in the program. At this point, the 
school district began raising questions about the 
students that continued to be sent to the out-of- 
community private day treatment settings and 
decided they wanted to pursue more information 
on these students. They also wanted to continue 
examining new placements into their public day 
treatment program, herefore, they added 50 new 
youth to the sample. They then added 50 new 
youth to the sample. These 50 youth were placed in 
either the above described public school day 
treatment program or in a private day treatment 
program. The school district wanted to examine 
similarities and differences in students in the 
public vs. private day treatment programs and 
monitor outcomes over time among both groups. 

Data collected in the spring of 1997 included 26 
of the initial 30 students along with 50 new youth 
added to this expanded sample (See Table 1 for total 
numbers for individual instruments. The number of 
instruments collected varies because it was not 
possible to obtain instruments for every youth). Ages 
ranged from 6 to 16 years, with a mean of 12.1 years 
(n=37) and 84.2% youth were male (n=38). About half 
were identified as African American (51.4%) and the 
other half as Caucasian (48.6%, n=37). 

Early intervention site. An elementary school 
in Southern Illinois collaborated with a local 
mental health center to provide early intervention 
mental health services for children identified by 
teachers as at-risk of placement in EBD. These 
prevention activities were provided by a mental 
health therapist and included: individual and 
group counseling, recreational activities, consulta- 
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tion to teachers and support to families. Baseline 
information was collected on 24 students through 
the winter of 1996-1997 and Time 2 information 
was collected on 14 students in the spring of 1997. 
(See Table 1 for total numbers of individual instru- 
ments. Like the day school treatment program 
sample, number of instruments varies). The 
average age of the youth was 9.8, ranging from 6 to 
13 years old (n=17). Eleven of the youth were 
male (n=17), and 12 out of the 15 children were 
Caucasian. 

Instruments and Analysis 

The instruments used for both sites are identi- 
fied in Table 1. The data were entered and tests were 
examined using the Statistical Package for Social 
Sciences (SPSS). These analyses examined both 
relationships between variables and changes over 
time. Descriptive statistics were used to provide 
basic information about the data. Relationships 
between variables were examined using correla- 
tional analyses, and cross-tabulation. Differences 
between groups were examined using independent 
t-tests, and cross tabulation. Changes over time in 
individuals’ clinical scores were analyzed using 
paired t-tests, and examining changes for groups of 
individuals in educational placement categories or 
out of home placement status were examined by 
using cross tabulation. 

Results 

School-based Day Treatment 

At the time of the Spring 1997 data collection 
for the expanded sample (public and private day 
treatment), almost one quarter (24%) of the youth 
were residing in a group or foster home placement 
under the guardianship of Illinois Department of 
Children and Family Services (DCFS). Thirty-five 
percent of the youth lived with one parent/ 
guardian. Only 16% of the youth lived with both 
parents/ guardians. Eight percent of the youth 
lived with a relative. 



The majority of youth were identified as having 
a primary behavioral disability by parents/guardian 
(75%), social workers (95%), and teachers (90%). 
However, more social workers reported youth as 
having an emotional or behavioral disability than 
parents/guardians = 2.97, d7=l, p=.081, n=31) 
and teachers and social workers tended to agree 
(X^ =10.37, d/'=l, p=.013, n=56). 

Parents reported that about half of youth had 
legal charges brought against them (n=36), while 
facilitators reported about one third of youth had 
legal charges brought against them (n=56). The 
majority of students had never been arrested with 
a conviction according to parent (8 out of 19) and 
facilitator (9 out of 48). 

The most common youth risk factors reported 
by parents/guardian included: below grade level 
achievement (54%), frequent suspensions/expul- 
sion/truancy (51.3%), and involvement with the 
legal system (41%), dangerousness to others 
(35.9%), history of substitute care (31%). Family 
risk factors reported by parents/guardians in- 
cluded history of family alcoholism (46%), 
unemployment (36%), and history of family 
violence (33%). 

Differences between Private Day School and 
Public Special School. Public and private day 
school program data collected in spring 1997 were 
compared. There were no significant differences 
between students in a private day placement 
(n=15) and a public special placement ( n=25) on 
the CBCL total problem score, internalizing, 
externalizing, and all sub-domain scores. 

There were no significant differences between 
students in a private day placement (n=23) and 
those in a public special placement (n=36) on the 
CAFAS total score. 

Youth served in the public day program (n=36) 
scored higher in the areas of Attention Problems 
(p=.05) and Aggression (p=.07) and Total External- 
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izing scores (p=.08) than youth served in a private 
day placement (n=25) according to teacher’s 
ratings on the TRF. 

Youth ratings of their aggressive behavior on 
the Youth Self-Report (YSR) were higher for youth 
in the special public school (n=22) than youth 
placed in the private day school (n=21; p=.04). 



Mental Health 

Early Intervention Program 

Data indicated that all of the youth lived with 
at least one parent, with the exception of one who 
lived with a relative (n=16). No youth had legal 
charges brought against them, or contact with the 
police as a result of a violation of the law. Twenty 
three facilitators indicated from a checklist the 
primary reasons for referral. These included: 
academic problems (n=19), non-compliance 
behavior (n=17), attention difficulties ( n=16), 
poor peer interaction (n=14). Parents and facilita- 



Table 1 

Instruments Collected at Baseline and Every Year Thereafter 



Instruments 


Day School Program - 
Number Collected 


Early Intervention 
Site — Number 




(Spring 1997) 


Collected (Winter 1996) 


Instruments Completed by Parents/Guardians 






Background Information 


38 


17 


Family Adaptiveness and Cohesiveness Scale (Olson, 1991) 


35 


15 


Family Needs (Dunst, Trivette & Deal, 1988) 


36 


16 


Services and Agency Involvement 


38 


17 


Youth and Family Risk Factors 


39 


17 


Satisfaction with Services 


35 


15 


Child Behavior Checklist (Achenbach, 1991a) 


40 


17 


Instruments Completed by Youth 






Youth Information Form 


59 


24 


Youth Self-Reports (Achenbach, 1991c) 


48 


3 


Instruments Completed by Teacher 






Educational Information Form 


62 


16 


Teacher Report Form (Achenbach, 1991b) 


61 


16 


Instruments Completed by Social Worker/Facilitator 






Background Information 


61 


23 


Restrictiveness of Living Environment Scale (Hawkins, 
Almedia, Fabry & Reitz, 1992) 


60 


23 


Services and Agency Information 


61 


23 


Youth and Family Risk Factors 


61 


23 


Child and Adolescent Functioning Assessment Scale 
(Hodges, Bickman & Kurtz, 1991) 


60 


24 
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tors identified a history of family alcoholism/ 
substance abuse as a family risk factor for 11 out of 
24 families. 

Twelve out of sixteen youth were placed in 
general education for 100% of the day (n=12), 
three youths enrolled in general education with 
consultation, and one youth attended a special 
education for less than 50% of the day. Nine of ten 
students maintained their educational placement 
from Time 1 (winter, 1996) to Time 2 (spring, 1997), 
and all of the youth attended school regularly at 
Time 1 and Time 2. 

At Time 1, classroom functioning data was 
available on 15 students. Areas of difficulty in- 
cluded completing class assignments on time 
(n=10) completing homework ( n=10), completing 
subjects with a passing grade (n=9), having 
friends (n=9), engaging in socially appropriate 
behavior with peers (n=ll), and engaging in 
socially appropriate behavior in unsupervised 
settings (n=ll). 

Teachers reported that seven students needed 
academic assistance beyond that which was expected, 
and nine youth required behavioral interventions 
beyond the regular classroom routine (n=16). Gener- 
ally, these students maintained their status of 
needing behavioral interventions beyond the regular 
classroom routine from Time 1 to Time 2. 

Teachers reported that Just over half of the 
youth had academic performance that was not 
commensurate with their ability (7 out of 13).. 
However, the majority (7 out of 11 youth) was 
performing at average to above average levels, and 
four of the youth were performing at a below 
average level. 

CBCL ratings on six of the youth fell within 
clinical ranges on the Total Problems Score (n=17). 
Five fell within clinical ranges on the Internalizing 
domain, and four fell within clinical ranges on the 
externalizing domain of the CBCL. The most 



frequently reported syndrome scale falling within 
clinical ranges was Social Problems. There was a 
decrease that approached significance in attention 
problems as rated by parents/guardians on the 
CBCL from Time 1 (M=6.00) to Time 2 ( M=5.14), 
p=.078. 

CAFAS results at Time 1 showed that 42% of 
the youth’s scores fell within clinical ranges 
(n=24) with an average of 32 and a range of 10-60. 
The average total CAFAS score at Time 2 was 15, 
with scores ranging from 0 to 30, with lower scores 
indicating better functioning (n=15). No youths 
scored in the clinical range at Time 2. 

Sixteen Teacher Report Forms (TRF) were 
completed at Time 1. Half of the ratings from 
teachers (n=8) fell within clinical ranges on the 
externalizing domain, while one student had a score 
that fell within the clinical range on the internalizing 
domain. Nine (56.3%) of the youth had scores that 
fell within clinical ranges on the Total Problems 
domain. TRF forms for eleven youth were com- 
pleted at Time 2. Two teachers’ ratings fell within 
clinical externalizing ranges and internalizing 
ranges. Three teachers’ ratings fell within clinical 
ranges on the Total Problems domain. 

Discussion 

In examining differences between youth in 
public and private day school programs, parents/ 
guardians do not appear to differ in their ratings of 
emotional and social functioning nor were there 
differences in clinical functioning reported by 
clinicians. However, teachers and youth reported 
higher rates of aggressive behavior among the 
public school group. This was an interesting 
finding since one might expect that the students 
sent outside of the community (i.e. private day 
treatment settings) might be more difficult to 
manage. District personnel report that they suspect 
a lack of clear parameters or decision-making 
mechanisms for when to send students to more 
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restrictive out-of-district placements contributes to 
this finding. More investigation is needed to 
further examine these differences. 

The data on successful returns to general educa- 
tion settings from the public day treatment settings 
was discussed with district administrators. The 
public day-treatment setting successfully returned 
over 50% of the students to less restrictive settings 
(with supports from the day treatment setting) in the 
first year. District personnel reported that the stu- 
dents in the newly developed public school program 
were returned to regular education settings faster 
and more effectively than students placed in the out- 
of-community private settings in the past. They 
began discussing possible program restructuring that 
may support more successful outcomes for greater 
numbers of students with EBD across the district. The 
experiences with this new program has suggested to 
the administrators that they may be able to improve 
school success for students with EBD by restructur- 
ing service options within their district settings. 

These discussions have led to a review of all district 
options for students with EBD (including those in 
self-contained classrooms in regular buildings) by 
district administrators. The restructuring of building- 
based options for service delivery that includes 
family supports, child-family team facilitators and 
more therapeutic options are being explored. 

Comparing the students from the prevention 
sample and the day treatment sample (combined 
public and private) provides opportunity for school- 
based program development discussion as well. 
Perhaps one of the most interesting findings is that 
although the samples were quite different with 
respect to age, restrictiveness of educational setting, 
and living situation, the risk factors reported and 
level of emotional and behavioral difficulty were 
remarkably consistent. For example, both day 
treatment and early intervention youth had multiple 
risk factors, and the most frequently reported risk 
factors included below grade level achievement and 
history of family alcoholism. The effects of the 



supportive approaches allowed children and youth 
(both prevention and day treatment group) to either 
move to less restrictive educational settings or 
maintain their current educational setting despite 
their level of clinical involvement. This suggests 
two issues for consideration. The first issue is that 
Improvement in clinical functioning may not be the 
only predictor of educational success. Factors such 
as teacher and family support may prove to be just as 
important. The second issue raised is the potential 
effect of early intervention through school-based 
mental health service delivery that actively supports 
families, teachers, and students. Future evaluation 
activities may help clarify the relationship of these 
factors to the children and youth outcomes over time. 

Initial analyses from these two school-based 
sites provide implications for future evaluation 
activities and implementation of system of care 
approaches in schools. One clear implications from 
an evaluation perspective is the strength of using 
participant driven evaluation if the data is to be 
useful for program development. Both of these 
school based sites determined their evaluation 
questions, reviewed instruments, made modifica- 
tions to instruments, collected data, and played an 
integral role in the evaluation process. Because of 
this, we had higher rates of instrument completion 
and accuracy in the data than many of our past 
evaluation efforts. Also, the data has been used 
locally to stimulate program development and has 
lead to expansion of system of care-based options 
in both locations 
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